Lessons Learned from Using an Online Platform to Conduct Large-Scale, Online Controlled Security Experiments with Software Developers
نویسندگان
چکیده
Security and privacy researchers are increasingly conducting controlled experiments focusing on IT professionals, such as software developers and system administrators. These professionals are typically more difficult to recruit than general end-users. In order to allow for distributed recruitment of IT professionals for security user studies, we designed Developer Observatory, a browser-based virtual laboratory platform that enables controlled programming experiments while retaining most of the observational power of lab studies. The Developer Observatory can be used to conduct largescale, reliable online programming studies with reasonable external validity. We report on our experiences and lessons learned from two controlled programming experiments (n>200) conducted using Developer Observatory.
منابع مشابه
Online Field Experiments: A Selective Survey of Methods∗
The Internet presents today’s researchers with unprecedented opportunities to conduct field experiments. Using examples from Economics and Computer Science, we present an analysis of the design choices, with particular attention to the underlying technologies, in conducting online field experiments and report on lessons learned. JEL Classifications: C93, H41
متن کاملMORF: A Framework for MOOC Predictive Modeling and Replication At Scale
The MOOC Replication Framework (MORF) is a novel software system for feature extraction, model training/testing, and evaluation of predictive dropout models in Massive Open Online Courses (MOOCs). MORF makes large-scale replication of complex machine-learned models tractable and accessible for researchers, and enables public research on privacy-protected data. It does so by focusing on the high...
متن کاملBotOnus: an online unsupervised method for Botnet detection
Botnets are recognized as one of the most dangerous threats to the Internet infrastructure. They are used for malicious activities such as launching distributed denial of service attacks, sending spam, and leaking personal information. Existing botnet detection methods produce a number of good ideas, but they are far from complete yet, since most of them cannot detect botnets in an early stage ...
متن کاملA NEW PROTOCOL MODEL FOR VERIFICATION OF PAYMENT ORDER INFORMATION INTEGRITY IN ONLINE E-PAYMENT SYSTEM USING ELLIPTIC CURVE DIFFIE-HELLMAN KEY AGREEMENT PROTOCOL
Two parties that conduct a business transaction through the internet do not see each other personally nor do they exchange any document neither any money hand-to-hand currency. Electronic payment is a way by which the two parties transfer the money through the internet. Therefore integrity of payment and order information of online purchase is an important concern. With online purchase the cust...
متن کاملGroup Learning and Performance in a Large-scale Software Project: Results and Lessons Learned
Background: Research on teams originated from the social sciences and brought a number of new topics into the repertoire of software engineering. Teams and teamwork are recognized for the promised benefits of i.e. increased performance. Performance is often linked to experience gains, and along with individual learning teamwork facilitates what is recognized as group learning. Aims: In this pap...
متن کامل